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INTRODUCTION: The Pediatric Emergency Department (PED) provides medical students with 
learning in a high-volume, fast-paced environment; characteristics that can be stressful for 
new students. Shadowing can improve transitioning, yet this alone does not facilitate students' 
development of independent medical care competencies. This study evaluates if third-year medical 
students' deliberate apprenticeship with senior residents increases students' comfort and patient 
exposure in the PED. 

METHODS: This study took place over the 2011-2012 academic year, and study participants 
were all third-year medical students during their pediatric clerkship rotation. This was a 
prospective educational intervention assigning students to randomized control blocks of deliberate 
apprenticeship (DA) intervention or control. DA students were paired with a senior resident who 
oriented and worked with the student, while control students were unpaired. All students completed 
a 20-question structured survey at shift end, which included questions about their perception of the 
learning environment, comfort with, and number of patient care responsibilities performed. We used 
independent Mann-Whitney and t-tests to compare experiences between the groups. Statistical 
significance was defined as p<0.05. We used the constant comparative method to qualitatively 
analyze students' comments. 

RESULTS: Response rate was 85% (145/169). Students also rated on 5-point Likert-scale their 
level of comfort with defined aspects of working in the PED. DA students (n=76) were significantly 
more comfortable obtaining histories (4.2 versus 3.8) and formulating differential diagnoses (3.9 
versus 3.4). DA students also performed more physical exams (2.9 versus 2.4). We categorized 
themes from the qualitative analysis of the students' comments about their PED experience. The 
titles for these themes are as follows: PED provides a good learning experience; uncertainty about 
the medical student's role in the PED; third-year medical students compete with other learners for 
teaching attention; opportunities provided to medical students for inclusion in patient care; personal 
knowledge deficits limit the ability to participate in the PED; PED pace affects learning opportunities. 

CONCLUSION: DA constitutes a feasible approach to the clinical learning environment that 
increases students' patient care experiences and may ease transitioning for undergraduate medical 
students to new clinical environments. [West J Emerg Med. 2014;15(4):424-429.] 
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Deliberate Apprenticeship Improves Student Experience 



INTRODUCTION 

The transition to new clinical environments can be a 
stressful experience for medical students. Lack of knowledge 
of their roles and lack of familiarity with their environments 
contributes to the anxiety-provoking nature of transitioning 
work places.' The pediatric emergency department (PED), 
in particular, is a high-volume, fast-paced environment that 
provides rich learning and training opportunities not found 
in other healthcare settings. Yet these very characteristics can 
be stressful for students not accustomed to this environment. 
Orientations, however, allow students to understand the 
"mechanics and processes of the healthcare team.- Expectations, 
such as how many patients to see, what information should be 
conveyed to the preceptor, and how much time to spend with 
each patient, are valued by the student.^ " 

Deliberate apprenticeship (DA) is the process of how 
learners gradually become part of a profession through 
progressively shared activities with established professionals. 
This concept of DA, in conjunction with orientation to learning 
objectives, is a key theoretical framework of how learning 
occurs.^ ' For instance, existing research shows that shadowing 
of current house officers can improve transitioning from the 
undergraduate to graduate medical training years, as well as 
comfort in a new practicing physician's role.'-^ Emergency 
department (ED) observational experiences have been shown 
to increase career interest in emergency medicine for first- 
year medical students by linking these students with well- 
entrenched learners in the field.' DA extends the principle of 
simply shadowing, by incorporating active involvement and 
participation as necessary steps to developing independent 
medical skills. Although there have been qualitative 
studies investigating the socialization and transition to new 
environments for house staff, there have been few educational 
interventions on how DA can improve the overall experience 
of the transition from the pre-clinical to the clinical years for 
undergraduate medical students. 

From our post-clerkship surveys conducted prior to the 
initiation of this study, it appeared that third-year medical 
students' ratings of their experience in the PED varied based on 
how actively the student was oriented and engaged during their 
shift. The absence of assignment responsibility for orientation 
for specific trainees leaves considerable variability in students' 
contacts with residents at different levels of training, as well 
as with attendings. We hypothesized that DA of third-year 
medical students with senior residents in the PED would 
improve medical students' overall experience in this setting 
compared to the usual practice, at this institution, of students 
just showing up for their shifts. 

METHODS 
Study Population 

The study population was the third-year class of 1 69 
medical students completing their pediatric clerkship for the 
academic year of 2011-2012. 



Study Setting 

This was a prospective educational intervention study 
conducted at the PED at a tertiary care children's hospital over 
a 12-month (rotation) period. This study was deemed exempt 
by the local institutional review committee. The pediatric 
clerkship at this institution consists of a total of eight weeks: 
four weeks in the inpatient hospital setting and four weeks of 
ambulatory experience. The PED experience consists of two 
eight-hour shifts that occur during the ambulatory period. 
Prior to the start of the overall pediatric clerkship, students 
receive a four-hour orientation. An online website contains 
additional information regarding various aspects of their 
pediatric clerkship. Included is a PowerPoint presentation 
about the PED that students are encouraged to review before 
their ED shifts. Specifics about the PED experience are not 
verbally presented during the four hour orientation. Instead, 
the overall orientation instructs students to simply "show-up" 
and "jump-in" during the PED shifts. 

Study Protocol 

Students were assigned in randomized blocks to DA 
(intervention) or control groups. The DA group consisted of 
students specifically paired with the senior residents whose 
shift schedule had the most significant overlap with the 
students' schedule. This resident was in charge of not only 
orienting, but also closely working with the student. These 
senior residents received an email with the date, shift time, 
student name, and specific instructions on how to orient their 
medical student. The control group consisted of students who 
received the traditional pediatric orientation: i.e. showed up 
for their shifts without a pre-assigned resident. The third- 
year medical students not paired with senior residents were 
not aware that other groups of students had specific pairings 
to residents. 

Outcome Measurements 

We evaluated this educational intervention by collecting 
medical students' perceptions about their experience 
during their PED shifts. Students were asked to complete 
a web-based (Qualtrics™) 20-question anonymous 
online instrument via email approximately one hour after 
completion of their first PED shift. Outcome data elicited 
from students included the number of patients seen, histories/ 
physicals performed, and procedures observed. Students 
also rated on 5-point Likert-scale their level of comfort 
with defined aspects of working in the PED. We also 
provided a section for written comments regarding their 
PED experience. Students were explicitly informed that their 
participation was voluntary and anonymous. 

Data Analysis 
Quantitative 

We analyzed the data using SPSS, Version 19 (Armonk, 
NY). Descriptive statistics for each variable were calculated. 
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Table. Differences between deliberate apprenticeship (intervention) of third-year medical students In a pediatric ED and control groups. 





Control mean 
(SD) 


Intervention 
mean (SD) 


p-value 


Comfort obtaining histories^ 


3.8 (1.0) 


4.2 (1.0) 


0.02* 


Comfort with physical examinations^^ 


3.7(1.1) 


4.1 (1.0) 


0.07 


Comfort with formulating a differential diagnosis^ 


3.4 (1.0) 


3.9 (0.9) 


0.001* 


Overall, comfort level working in the pediatric emergency department^ 


3.7(1.1) 


4.0 (1.0) 


0.11 


Overall teaching^ 


3.4(1.0) 


3.4 (1.1) 


0.88 


Overall experience^ 


3.4(1.0) 


3.5 (1.1) 


0.40 


Number of patients seen 


5.8(1.7) 


5.5 (1.6) 


0.31 


Number of histories performed 


2.6 (1.4) 


3.0 (1.4) 


0.08 


Number of physical exams performed 


2.4(1.3) 


2.9 (1.5) 


0.03* 


Number of procedures observed 


2.0 (1.0) 


2.0 (1.2) 


0.76 



*p<0.05, tLlkert-type scale very uncomfortable (1 ) to very comfortable (5). 



We used independent Mann- Whitney analyses, for ordinal- 
level ratings of comfort and overall PED teaching and 
experience, to compare the experiences of students in the 
DA and the control groups. We compared the number of 
patients that DA and control students saw and procedures 
they observed, as well as the number of histories/physical 
examinations they independently performed, using 
independent t-tests. We studied this over an academic year 
to determine empirically whether differences emerged over 
time, i.e. early versus late rotation periods with analysis 
of variance over the 12 rotations. Statistical significance 
was defined as p<0.05. We used a Cohen's d to calculate 
effect size; based on a population of 169 students, with 95% 
confidence level, effect size 0.5, power level of 0.8 and 80% 
response rate, this was a total of 102 students. 

Qualitative 

Using the constant comparative method, consistent with 
grounded theory, the comments provided by the medical 
students were independently read and coded for emergent 
themes by three investigators, who were blinded to the 
comment status as generated by the DA or the control group.'" 
The analysis proceeded iteratively until all comments were 
coded into themes consistently across reviewers. 

RESULTS 
Quantitative Results 

A total of 145 out of 169 (85%) students responded to the 
online survey instrument. Figure illustrates the breakdown 
of individuals completing the survey. Most (74.1%) of the 
students reported that they had read the online orientation 
PowerPoint prior to their PED shift. The majority of students 
(85%) reported receiving an individual orientation during 
their PED shift. Table summarizes the findings comparing 
the DA and control groups. At a statistically significant level, 
DA students were more comfortable with taking histories and 
with creating a differential diagnosis. In addition, DA students 



performed significantly more physical examinations. There 
was no significant difference in students' rating of their overall 
PED experience in comparing mean ratings of the two groups. 
No statistically significant difference occurred in comparing 
mean number of professional care responsibilities across the 
12 rotations during the academic year. 

Qualitative Results — Themes from Student 

There were 58 comments from the students. Six themes 
emerged from the analysis of these comments. A summary 
of each theme, noting the percent of students contributing 
comments categorized within the theme category, is provided. 
The themes were as follows: PED provides a good learning 
experience (39.7%); uncertainty of the medical student's 
role in the PED (27.6%); opportunities provided to medical 



SentSurvey 
n = 169 



Completed 

Survey 
n = 145 (85%) 

7^ 



Control 
n = 76 (52%) 



Deliberate 
Apprenticeship 
n = 69 (48%) 



Figure. Study population. 
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students for involvement/inclusion in patient care (12.1%); 
third-year medical students compete with other learners for 
teaching attention (3.4%); personal knowledge deficits limit 
the ability to participate in the PED (3.4%); and PED pace 
affects learning ability (13.8%). A description of each theme, 
with illustrative quotations, is provided below. 

PED Provides a Good Learning Experience 

Many medical students characterized the PED as a 
valuable part of their training. One student commented, "I 
think that it is an important experience because you get to 
see/hear very pertinent findings that are often not present in 
clinic (ex: real nasal flaring and retractions or even in my case 
a patient in status (control)." Faculty and residents were also 
individually acknowledged as contributing to the medical 
students' having an overall good experience. For instance, one 
student wrote, "Dr. [X] was GREAT, so was [Y] the resident. . . 
they were great teachers and definitely included teaching as 
well as feedback (DA)." 

Uncertainty of the Medical Student's Role in the PED 

Many medical students reported that faculty and house staff 
did not know how to incorporate them into the flow of patient 
care. One medical student commented, "It was a httle hectic 
when I arrived and I wasn't informed of exactly what I would 
be expected to do — nobody seemed to know what my role 
was exactly and whether I should see patients on my own, etc 
(control)." Students reported that if expectations were provided, 
they would be better suited to participate in this environment. 
One student wrote, "At the beginning nobody looked like they 
were expecting to see an M3 show up for the shift; nor were we 
given instructions at peds orientation to ask for Dr. X (attending, 
resident, etc.) on arriving at the ED. Once I introduced myself 
and got setup with a resident everything was great, but knowing 
who on staff to talk to when you get there would make things 
run smoother" (control). 

Opportunities Provided to the Medical Student for 
Involvement/Inclusion in Patient Care 

Students reported that when they were afforded the 
opportunity to participate directly in patient care, their overall 
learning improved. A student commented that, "[I] loved 
my time in the ED, as I was able to jump from patient to 
patient,. . .and really able to see whatever patients I wanted 
to. . .[I] loved working with residents who let me take the 
history myself, and gave me more autonomy in the physical 
exam, ordering tests, discharging patients" (DA). 

Third Year Medical Students Compete With other 
Learners for Teaching Attention 

Medical students are not the only trainees in the PED. 
Medical students characterized this presence of other levels 
of learners as working against medical students seeing 
patients. One of the students wrote that, "There were so many 



providers (1 intern, 1 resident, 1 M4 and a faculty member 
plus myself). . .1 didn't get to do an interview, I barely talked to 
a patient all day" (control). 

Personal Knowledge Limit the Ability to Participate in 
the PED 

Medical students felt that their own knowledge gaps, as 
well as being new to the clinical environment, affected their 
ability to participate in the PED. One student exclaimed that, 
"as an early M3 who hasn't yet completed internal medicine 
I felt woefully under-prepared to work with most ER patients 
and without having done anything except two weeks of well 
child exams on outpatients I feel even less prepared to take 
histories on pediatric patients" (control). 

PED Pace Affects Learning Ability 

The PED is often a hectic place, yet there are also 
times when there is low volume. One student expressed this 
experience of environmental variability by writing that this is 
a "busy place even when it's not 'busy'" (control). Another 
student reported that "both shifts I was there, the PED was 
pretty quiet, so I didn't get to see as many interesting things as 
I otherwise would have" (DA). 

DISCUSSION 

Medical educators have recognized that traditional medical 
education may insufficiently prepare students for the transition 
from preclinical to clinical education." '^ One response to this 
challenge has been the development of transitional courses.'"'* 
We know that the PED is a rich learning environment and 
that some EM educators have noted should be incorporated 
into early training. "''In this study, we tested if DA for third 
year medical students would improve their experience in the 
PED. Yet, the qualitative themes show that students found the 
PED to be problematic in terms of uncertainty of role, level of 
inclusion, and self-perceived knowledge deficits. We found that 
our DA intervention helped in these domains by perfonnance 
of more physical examinations and increased comfort with 
taking histories and formulating differential diagnoses. 

The theory of situational learning has been prominent 
in identifying the active social methods that learners and 
professionals use in DA to facilitate the progressive awarding 
of participation and autonomy of learners. In DA, the 
apprentice student observes the master demonstrating how 
to do different parts of the task.''-'' The master deliberately 
makes the target processes visible, often by explicitly showing 
the apprentice what to do, for example describing how they 
evaluate a patient and thinking out loud through the clinical 
reasoning. This extends what has previously been described as 
simply "orientation," "shadowing," to "legitimate peripheral 
participation in a community of practice.""' By pairing students 
with senior residents who not only oriented, but also closely 
engaged the students, DA enhanced students' perception of 
their learning. Further, students who were actively engaged 
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during their PED shifts commented on having better individual 
experiences. In particular, comments on students' level of 
inclusion with residents and/or faculty allude to the idea that 
DA increases participation in the unique clinical environment 
of the PED. 

As medical school curricula evolve, there is a move 
towards earlier emergency medicine experiences." '* In light 
of this, it is important to know and address some of the pitfalls 
of third-year students in the potentially chaotic PED. One 
key issue is uncertainty with their role, with some students 
requesting a clear orientation. This study also showed that the 
majority of students reported receiving an orientation to the 
PED, whether or not they were assigned to a senior resident. 
Even so, they remained uncertain of their role. This concept 
of providing information to a student, yet the student not 
perceiving that it has been provided, is not uncommon. For 
example, one study on feedback for medical students showed 
statistically significant differences in the regularity of feedback 
provided by faculty as compared to that perceived by the 
student." Another potential reason for this discrepancy is that, 
although they were oriented to the PED, they did not receive 
ftirther instruction for the duration of their shift or simply 
could not independently perform without direct instructions. 
This is best put in the words of one student who wrote, "I did 
appreciate one of the residents had me look up a treatment 
regimine [sic] for one of the cases we saw and present that 
to her." Such directed learning may be necessary for some 
individuals to have an overall better experience in the PED. 

The themes culled from student comments help us 
to understand the experience of students in the PED. We 
recognize that we need to be more intentional in orienting and 
engaging students in clear roles, inclusion in patient care and 
learning. DA may help with many of these issues but further 
study is warranted. 

LIMITATIONS 

Although the senior residents did receive an email with 
specific instructions on what/how to orient their students, we 
studied only the medical students' experiences. It is therefore 
possible that some of the seniors were not able to orient their 
pre-assigned medical students during their shift, or oriented 
students to whom they were not paired, raising the issue of 
the fidelity of the study with its intended implementation. The 
study did not explore the extent to which participation in DA 
might have enhanced faculty engagement. In our institution, 
as is true for other clerkships, the length of the PED rotation 
is limited. Our medical students completed two shifts in 
the PED, thus limiting assessment of improvement over 
time. Furthermore, DA students were not necessarily paired 
with the same senior resident each shift. In addition, not all 
students entered comments. Although the number of students 
participating in the study enabled detecting a statistically 
significant difference between the DA and control groups 
at alpha=0.05, the effect size was moderate, with some 



overlapping confidence intervals. Furthermore, we made 1 0 
comparisons between the DA and control group in regards 
to comfort levels in various domains, as well as number 
of medical skills completed. These multiple comparisons 
may have led to a Type 1 error. Moreover, in seven of these 
comparisons, there were no differences between our groups, 
thus the full extent of DA needs to be further elucidated. If 
the alpha levels are fiirther adjusted for multiple comparisons 
with the conservative Bonferroni Correction, which divides 
alpha by the number of comparisons made, only comfort 
in making a differential diagnosis remains as a comparison 
representing a statistically significant difference between 
the groups. Finally, the study was performed at a single 
institution, thus limiting generalizability. 

CONCLUSION 

This educational intervention study extends existing 
research on DA by providing insights into the kinds of 
outcomes such intentional allocation of resources may achieve. 
This study adds to our current professional discourse on the 
move from time-based to competency-based education. Much 
of the discussion about this move focuses on the need to be 
more intentional in designing education promoting cognitive 
and procedural competence. This randomized controlled block 
study, using mixed quantitative and qualitative approaches, 
builds on and extends this literature by focusing on intentional 
approaches to student socialization to the work-place learning 
site. The intervention evaluated represents a theoretically- 
grounded and feasible course of action amenable to being 
adopted in other clinical learning sites. 

The emergent themes from analysis of the students' 
comments illuminate how active engagement can improve 
experience. Although students' ratings of their overall 
experience in the PED did not differ across the two groups, 
students who underwent DA performed more physical exams, 
and had greater comfort in obtaining histories and formulating 
a differential diagnosis. Above all, DA is a novel, simple, 
and feasible technique to ease the transition to new clinical 
environments for undergraduate medical students. 
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